1. Field of the Invention
The present invention relates to a technique for calibrating the measurement results of a sensor which measures the three-dimensional position and orientation.
2. Description of the Related Art
Studies about mixed reality presenting a result obtained by superimposing text and CGs on a physical space have been extensively made. An image display apparatus that presents mixed reality can be implemented as an apparatus which superposes and renders, onto an image sensed by an image sensing device, an image generated according to the position and orientation of this image sensing device.
In order to implement such image display apparatus, the relative position and orientation between a world coordinate system defined on the physical space and a camera coordinate system defined on the image sensing device need to be measured in real time. For example, a case will be examined below wherein a virtual object is superimposed at a predetermined position in a physical environment such as that in a room or on a table. In this case, a world coordinate system is defined on an appropriate place (e.g., the floor surface of a room or the table surface) in that environment, and the position and orientation of the image sensing device on the world coordinate system can be measured. As means for attaining such measurements, three-dimensional (3D) position and orientation sensors such as a magnetic sensor, optical sensor, ultrasonic sensor, and the like are popularly used.
In general, measured values output from the 3D position and orientation sensor are the position and orientation of a survey point (measurement point) on a coordinate system uniquely defined by the sensor (to be referred to as a sensor coordinate system hereinafter), and are not those of the image sensing device on the world coordinate system. For example, taking the magnetic sensor FASTRAK available from Polhemus as an example, the position and orientation of a receiver on a coordinate system defined by a transmitter are obtained as sensor outputs.
Therefore, the sensor measured values cannot be used as the position and orientation of the image sensing device on the world coordinate system as is, and they need to undergo some calibration (transformation). More specifically, the sensor measured values need to undergo the following two coordinate transformations.
The first coordinate transformation transforms the position and orientation on the sensor coordinate system into those on the world coordinate system. Information required to implement the first coordinate transformation (to be referred to as first coordinate transformation information hereinafter) includes the relationship between the positions and orientations of the world coordinate system and sensor coordinate system. This information is expressed by, for example, the position and orientation of the sensor coordinate system on the world coordinate system.
On the other hand, the second coordinate transformation transforms the position and orientation of the survey point to those of the image sensing device. Information required to implement the second coordinate transformation (to be referred to as second coordinate transformation information hereinafter) includes the relationship between the positions and orientations of the image sensing device and survey point. This information is expressed by, for example, the position and orientation of the survey point on a coordinate system defined by the image sensing device (to be referred to as a camera coordinate system hereinafter). In this specification, these pieces of information required to transform the sensor measured values into the position and orientation of the image sensing device on the world coordinate system will be generically referred to as “calibration information” hereinafter.
Patent reference 1 discloses a method of easily measuring calibration information of the 3D position and orientation sensor attached to the image sensing device. This method is implemented by the following sequence.
(1) Indices whose 3D coordinates (to be referred to as world coordinates hereinafter) on the world coordinate system are given are allocated or set in a scene.
(2) The measured values of the 3D position and orientation sensor and images sensed by the image sensing devices at a plurality of different viewpoint positions are simultaneously acquired.
(3) The indices are detected from the acquired images, and actually measured values (to be referred to as observation coordinates hereinafter) of image coordinates of the detected indices are acquired.
(4) Approximate values of the first coordinate transformation information and second coordinate transformation information as calibration information of the 3D position and orientation sensor are input, and are used as initial estimated values.
(5) As for the respective detected indices, the image coordinates (to be referred to as projected coordinates hereinafter) of the indices are calculated based on their 3D coordinates, the measured values of the 3D position and orientation sensor, and the current estimated values of the calibration information, and errors between the projected coordinates and observation coordinates are calculated.
(6) The estimated values of the calibration information are corrected to minimize the sum total of the errors.
(7) The processes (5) and (6) are repeated until the correction process (6) converges.
With this method, by providing rough values of the calibration information, high-precision calibration information can be obtained without using any special calibration jig.
Patent reference 1 discloses a simple measure for avoiding manual input operations of the approximate values of the calibration information. In this measure, some patterns as the approximate values of the calibration information are prepared in advance.
These approximate values are set as initial estimated values in turn to execute the processes (5) to (7), and the calibration information upon convergence of the correction process is selected.
[Patent Reference 1] Japanese Patent Laid-Open No. 2003-269913
However, with the method of patent reference 1, the approximate values of the first coordinate transformation information and second coordinate transformation information as the calibration information to be calculated need to be input, resulting in cumbersome operations. If the input approximate values are far from actual values, the correction calculations do not converge, and the calibration information cannot be obtained.
With the method of preparing some patterns in advance as the approximate values of the calibration information, when the prepared approximate values are far from actual values, the correction calculations do not converge, and the calibration information cannot be obtained. In particular, the relationship between the positions and orientations of the world coordinate system and sensor coordinate system as the first coordinate transformation information has a very high degree of freedom, and it is difficult in practice to prepare initial values that can converge in all cases.